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EXECUTIVE  SUMMARY 


General  Electric  Global  Research  and  Massachusetts  Institute  of  Technology  (MIT)  have  built  and 
demonstrated  a  crowd-driven  ecosystem  for  evolutionary  design  (CEED)  to  support  DARPA’s 
“vehicleforge.mil”  program  to  revolutionize  modern-day  design  and  manufacturing.  Through  this 
platform,  DARPA  is  looking  to  enable  a  global  community  of  experts  to  design  and  rapidly  manufacture 
complex  systems  such  as  military  vehicles,  aviation  systems,  and  advanced  medical  devices.  These  cyber¬ 
physical  systems  can  take  decades  to  develop.  The  primary  goal  of  this  program  is  to  dramatically  reduce 
that  timeline. 

The  CEED  crowdsourcing  platform  will  connect  data,  design  tools,  and  simulations  in  a  collaborative 
environment  to  accelerate  the  design  of  highly  complex  industrial  systems.  The  CEED  vehicle  forge 
platform  allows  designers  to  team  up  to  develop  projects  concurrently.  Developers  from  different  spaces 
will  be  able  to  form  design  communities  and  create  a  common  project  that  will  allow  them  to  manage 
processes  as  a  team  and  track  changes  and  updates. 

Furthermore,  the  CEED  platform  provides  a  marketplace  where  contributors  can  choose  to  expose  their 
ideas  to  the  public  either  as  open  source  or  as  IP  protected  services.  One  unique  feature  of  CEED  is  the 
incorporation  of  MIT’s  Distributed  Object-based  Modeling  Environment  (DOME) — a  simulation 
“engine”  that  contributors  use  to  plug  in  and  test  simulated  components.  Users  can  attach  simulation 
services  to  explore  the  behavior  of  complex  systems,  and  to  predict  problems  earlier  to  get  a  better  design 
faster.  In  his  New  York  Times  article,  Steve  Lohr  states  that  “there  are  plenty  of  computer-aided  software 
tools  used  in  design  and  manufacturing.  But  the  software  is  often  difficult  to  use  and  expensive”  [Lohr 
2012].  With  DOME,  the  vehicleforge.mil  program  will  “allow  solo  inventors  or  small  teams  to  tap  into 
those  capabilities.  A  vehicle  body  and  chassis  design,  submitted  as  code,  could  be  plugged  into  the 
vehicleforge.mil  platform  and  tested  for  aerodynamics  in  a  virtual  wind  tunnel”  [Lohr  2012], 

The  exposed  design  models,  or  models  available  for  review  in  this  open  community,  will  experience 
market  pressures  that  ultimately  allow  for  the  best  designs  to  emerge.  The  CEED  platform  also  embeds 
social  media  connections  to  maximize  crowd  engagement.  Within  such  an  environment,  experts  and  non- 
traditional  contributors  (crowd)  can  design  individual  components  or  entire  systems  by  reusing,  remixing, 
and  building  on  designs  prepared  by  others.  These  designs  will  continue  to  evolve  through  a  series  of 
reiterative  design  loops  that  produce  crowd-driven  selection  pressure  including  design  and  performance 
tests. 
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TEAM 


The  CEED  team’s  inspiration  comes  from  MIT’s  DOME  research  initiative.  In  1995,  David  Wallace,  a 
professor  in  MIT’s  CAD  lab,  coined  the  concept  of  “service  marketplace”  as  a  paradigm  shifting  way  for 
predictive  integrated  system  modeling  in  the  design  of  complex  products.  Over  the  next  10  years,  a 
distributed  computational  infrastructure — DOME — has  been  developed  as  an  enabling  technology  for  this 
design  service  marketplace  vision.  Qing  Cao,  now  a  researcher  at  GE  Global  Research,  was  one  of  the 
key  personnel  on  the  DOME  project  while  she  worked  toward  her  Ph.D.  For  CEED,  the  team  is 
expanding  DARPA’s  original  requirement  of  a  project  hosting  and  management  website  to  include 
DOME’S  service  marketplace,  which  is  a  key  differentiator  and  enabler  for  design  ecosystems. 

Qing  and  David  are  the  Principal  Investigators  of  this  CEED  activity.  The  GE  Global  Research  team  has  a 
team  of  computer  scientists  and  software  engineers  that  bring  in  extensive  expertise  in  software  research 
and  production  software  development.  Benjamin  Beckmann  is  leading  the  federated  authentication  and 
cloud  deployment  effort.  Thomas  Citriniti  is  in  charge  of  the  overall  software  architecture  and 
infrastructure.  More  information  can  be  found  in  the  team  bios  section. 
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TASK  OBJECTIVES 


1.1.  Research  Goals 

Our  research  goals  are: 

•  Research  and  develop  key  enabling  technologies  for  crowdsourcing  the  design  of  complex 
systems. 

•  Provide  a  web-based  collaborative  design  environment  enabling  the  evolution  of  CEM  systems 
within  a  crowd-accessible,  service  exchange  marketplace. 

•  Investigate  mechanisms  to  attract  a  crowd  of  productive  experts,  non-experts,  and  companies  in  a 
cooperative  and  rewarding  ecosystem. 

•  Facilitate  design-cycle  incorporation  of  real-world  performance  feedback  received  from  dynamic, 
competitive,  and  hostile  environments. 

•  Investigate  mechanisms  to  protect  the  intellectual  property  (IP)  and  value  within  a  collaborative 
crowdsourcing  environment. 

•  Address  traditional  system  integration  difficulties  associated  with  scale,  complexity,  rate  of 
change,  heterogeneity,  and  proprietary  information. 

1.2.  Approach 

In  order  to  answer  DARPA’s  call  for  a  collaborative  crowdsourcing  environment  to  democratize  the 
design  process  of  complex  cyber-electromechanical  (CEM)  systems,  the  team  developed  CEED,  which  is 
a  web-based  collaborative  environment  to  support  the  crowdsourcing  of  CEM  systems. 

Such  an  environment  allows  designers  to  team  up  to  develop  projects  concurrently.  Developers  from 
different  locations,  backgrounds,  and  pedigree  will  be  able  to  form  design  communities  and  create  a 
common  project  space.  This  space  will  allow  them  to  manage  processes  as  a  team  and  track  changes  and 
updates  on  their  project. 

Furthermore,  the  CEED  platform  provides  a  marketplace  where  contributors  can  choose  to  expose  their 
ideas  to  the  public  either  as  open  source  or  as  IP  protected  services.  The  exposed  design  models,  or 
models  available  for  review  in  this  open  community,  will  experience  market  pressures  that  ultimately 
allow  for  the  best  designs  to  emerge.  This  will  mirror  how  designs  are  vetted  in  the  open  market  today, 
but  will  work  faster.  It  promotes  an  active  evolutionary  process  to  source  distributed  design  components, 
and  submit  them  to  aggressive  functional  and  system  testing  on-the-fly  in  a  reiterative  biomimetic 
“hardening  process.” 

The  CEED  platform  also  embeds  social  media  connections  to  maximize  crowd  engagement.  When 
completed,  the  software  developed  under  the  contract  will  be  open-sourced  and  used  to  support  portions 
of  vehicleforge.mil. 
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2.  DIFFICULTY  DEMOCRATIZING  CEM  DESIGN,  DEVELOPMENT,  AND  SYNTHESIS 

A  collaborative  crowdsourcing  environment  to  enable  the  democratization  of  CEM  design  is  very 
different  from  the  democratization  of  software  design.  Several  key  challenges  exist. 

2.1.  Uncover  mechanisms  to  promote  “grassroots'1  style  crowdsourcing  of  CEM  design 
and  development 

Although  open-source  crowdsourcing  environments  have  significantly  benefited  software  development, 
the  art  of  crowdsourcing  for  innovation,  quality,  and  cost  efficiency  has  not  been  successfully  translated 
into  hardware  design  and  development,  especially  for  CEM  systems.  Motivations  for  skilled  individuals 
and  companies  to  participate  (e.g.,  altruism,  community  sense,  free-riding,  effort,  etc.)  need  to  be 
explored  to  build  mechanisms  to  attract  and  sustain  a  crowd  of  CEM  designers. 

2.2.  Deal  with  systems-level  integration  and  testing  challenges 

Typical  CEM  design  involves  multi-domain,  multi-physics  modeling,  analysis  and  simulation.  Designers 
of  these  systems  use  complex  chains  of  heuristics  to  model  the  interaction  between  different  components. 
It  is  the  intent  of  this  work  to  develop  collaboration  and  design  techniques  to  enable  advancement  of 
crowd-oriented  design.  To  make  this  possible,  any  crowd-sourced  CEM  collaboration  system  must  be 
responsive  to  the  specific  challenges  of  the  system  domain. 

Complex  system  integration  of  any  kind  relies  on  well-defined  and  understood  interfaces  between  system 
components.  The  development  of  software  systems  is  simplified  because  well-defined  interfaces  are 
readily  available  for  integration.  By  comparison,  CEM  design  is  extremely  difficult  because  of  the 
integration  of  heterogeneous  models,  data,  computations,  and  tools  involved  is  complicated.  This  is 
partially  due  to  the  size,  complexity,  and  evolving  nature  of  the  models  in  hardware  design.  It  is  also  due 
to  the  need  for  different  tools,  data  management,  and  representations;  and  to  the  fact  that  all  required  data 
are  not  globally  available,  for  logistical  or  proprietary  reasons  (Cutkosky  et  al.  1996).  Generic  integration 
platforms  that  could  support  globally  optimizing  the  system-level  CEM  design  and  testing  are  still  to  be 
developed. 

2.3.  Support  biomimetic  "hardening  of  the  design"  process 

Product  design  and  development  is  a  highly  iterative  process.  The  combination  of  intended  or  accidental 
variations,  from  iteration  to  iteration,  commonly  exposes  design  flaws  and  occasionally  reveals 
innovation.  This  process,  in  the  hands  of  the  crowd,  can  enhance  the  creation  of  robust  designs,  exceeding 
limitations  of  processes  that  solely  depend  on  deterministic  models  or  elaborate  specifications. 

2.4.  Enable  design  reuse  of  both  open-source  and  proprietary  modules 

Typical  open-source  crowdsourcing  participants  are  drawn  from  a  collection  of  amateurs,  volunteers,  and 
experts  from  small  business  (Howe  2006).  Unlike  software  systems,  however,  the  design  of  a  complex 
CEM  system  often  involves  the  need  for  intensive  engineering  expertise,  expensive  design  and  analysis 
tools,  and  specialized  test  facilities.  A  majority  of  these  types  of  professionals  with  both  the  desired 
expertise  and  training,  and  with  access  to  the  required  tools  and  facilities,  resides  within  the  proprietary 
walls  of  industry.  Uncontrolled  release  of  proprietary  knowledge  in  the  form  of  documentation, 
schematics,  or  computational  models  is  currently  inappropriate  without  a  major  change  in  how  this 
information  can  be  safely  shared  with  its  value  retained.  Legal  restrictions  aside,  existing  compensation 
and  cooperation  models  and  other  technical  limitations  will  negatively  impact  the  crowd  participation 
paradigm  if  open-source  models  are  the  only  way  to  contribute. 
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2.5.  Provide  easy  access  to  design  tools 

The  cost  of  the  design  tools  represents  a  substantial  challenge.  A  designer  must  be  afforded  suitable  tools, 
representations,  simulations,  heuristics,  or  models  to  fulfill  design  requirements  and  handle  complexities. 
However,  this  affordance  drives  designers  toward  specialized  products  whose  availability  and  costs  limit 
the  design’s  potential  and  restricts  its  community  of  users.  This  gap  must  be  overcome  to  democratize 
CEM  design. 


3 


3. 


GENERAL  METHODOLOGY 


3.1.  Background 

The  CEED  platform  is  analogous  in  many  ways  to  project  management  systems  like  SourceForge 
(http://sourceforge.net/)  and  GitHub  (https://github.com/).  Traditionally,  these  types  of  systems  allow 
globally  distributed  individuals  to  collaborative ly  develop  software.  However,  the  modernization  of 
computer  aided  drafting  (CAD),  engineering  (CAE),  and  modeling  (CAM)  has  allowed  an  engineering 
team  to  be  globally  distributed.  Furthermore,  computer  aided  virtual  environments  allow  the  engineers  to 
share  and  iterate  on  work  products  to  produce  robust,  cost  effective,  and  essentially  more  valuable 
designs. 

Project  management  platforms  designed  for  software  development  enable  a  transparent,  iterative  design 
process.  This  process  has  proven  successful  for  software  development,  and  its  application  to  traditional 
engineering  is  promising,  even  for  large-scale  designs. 

However,  a  software  development  project  management  system  would  not  be  adequate  for  engineering 
without  modification  and  integration  into  an  engineer’s  daily  work  process.  Adding  another  system  to  an 
engineer’s  work  process  only  adds  value  if  it  enhances  the  engineer’s  ability  to  complete  the  assigned 
work. 

The  CEED  Platform  is  architected,  designed,  and  built  to  seemly  integrate  with  existing  engineering 
practices.  It  allows  for  the  sharing,  integration,  storage,  and  retrieval  of  engineering  artifacts,  including 
parametric  models,  test  results,  and  other  data  essential  to  the  engineering  process. 

To  achieve  successful  delivery  of  the  CEED  platform  within  the  one-year  time  frame,  the  team  leveraged 
an  agile  development  process.  This  process  was  driven  by  user  stories  developed  collectively  by  computer 
scientists,  engineers,  and  program  managers.  These  stories  helped  to  identify  common  themes  that  are 
present  throughout  the  platform,  and  ambiguous  concepts  that  could  be  problematic  for  a  cross 
disciplinary  team. 

3.2.  Technical  Approach 
3.2.1.  Overview 

The  CEED  platform  was  designed  and  developed  as  a  crowdsourcing  collaborative  environment  to 
democratize  the  design  process  of  complex  CEM  systems. 

Figure  1  is  an  overall  concept  chart  of  the  CEED  platform. 


4 


Crowdsourcing 

Platform 


^  crowd;  p 

Individuals  ond 

Organ  Nations 


Publish/  Subscribe 


Ad=Hoc  Modeling  and 
Simulation  Tools 


Service  Marketplace 
I  Ecosystem] 


Collaboration 

Wiki,  Forums, 
Mailing  Lists 
Etc. 


Services 


DOME 


H  osti  n  g  &  Col  I  aborati  on  ^ 

hdei  *  Project  Space 

osjtaiy  s  Real-Time  Design 

Verification 
•  Version  a 

Wm  Control 


"Bu^", 


Models 


Figure  1.  Conceptual  overview  of  the  CEED  platform 


3.2.2.  Key  Features 

3.2.2.1  Project  Hosting  and  Management 

An  off-the-shelf  open-source  project  management  platform  can  serve  as  a  collaboration  environment  that 
allows  geographically  distributed  contributors  to  collaborate  on  a  common  goal.  The  platform  assists  in 
the  management  of  design  and  development  efforts  by  providing  a  variety  of  collaboration  tools 
integrated  into  a  single  platform  where  a  project  can  be  created  and  the  collaborative  tools  are 
immediately  available.  Project  discussions  are  supported  through  forums,  mailing  lists,  message  boards, 
and  micro  blogging  streams.  Furthermore,  generation  of  project-specific  content,  such  as  documentation 
or  tutorials,  is  supported  in  a  project’s  wiki.  Lastly,  the  platform  provides  an  integrated  version  control 
repository.  These  collaboration  tools  support  the  self-organization  and  archival  of  project-specific 
information. 

3.222.  Integrated  Simulation  Environment 

CEM  design  platforms  need  to  address  system-level  integration  of  components  represented  by  cross¬ 
platform,  multi-domain,  multi-physics  models,  analysis  and  simulations.  MIT’s  DOME  is  a 
computational  infrastructure  for  this  purpose.  DOME  provides  a  foundation  of  service  definitions  and 
mechanisms  that  allow  designers  to  define  how  their  model  or  process  can  be  described,  connected,  and 
executed  through  a  distributed  service  oriented  architecture  (SOA).  DOME  enables  a  platform  upon 
which  a  community  can  build  integrated  design  scenarios  with  crowd  contributed  computational 
resources. 

322.3.  Distributed  Service  Marketplace 

CEED  augments  the  platform  with  a  distributed  design  service  marketplace.  The  distributed  service 
marketplace  allows  individuals  and  companies  to: 

•  Exchange  models,  services,  and  opportunities. 
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•  Integrate  nontraditional  team  members  who  can  freely  express  opinions,  vote,  test,  and  distribute 
information. 

•  Build  a  reputation  as  a  CEM  expert. 

•  Foster  a  healthy  ecosystem  through  controlled  design  selection  that  penalizes  poor  designs, 
rewards  robust  ones,  and  potentially  reveals  unforeseen  and  unplanned  features. 

•  Facilitate  the  democratization  of  hardware  design. 

3.2.2A  Authentication  and  Authorization 

Additionally,  the  CEED  system  is  augmented  with  export  controls,  private  branching,  and  federated 
authentication  and  authorization  capabilities.  These  features  support  user  and  project  security.  The 
security  solution  is  built  on  freely  available  and  established  open-source  software  such  as: 

•  Shibboleth:  http://shibboleth.net/ 

•  OpenLDAP:  http://www.openldap.org/ 

•  Kerberos:  http^/web.  mit.  edu/kerberos/ 

Users  from  multiple  organizations  will  be  able  to  gain  access  to  specific  projects  by  authenticating  to  their 
home  institution’s  identity  provider,  which  in  turn  provides  a  limited  set  of  user  attributes  required  for 
authorization. 

3.2.3.  Software  Architecture 

3.2.3. 1  Architectural  Representation 

The  principal  objective  of  the  vehicleforge.mil  effort  is  to  generate  an  open  source  development 
collaboration  environment  and  a  website  for  the  creation  of  large,  complex,  cyber-electro-mechanical 
systems  by  numerous  unaffiliated  designers.  In  short,  this  is  a  large  forge-based  site  where  designers  can 
work,  post,  and  design  CEM  systems  in  a  crowd  sourced  environment. 

To  satisfy  the  requirements  for  such  a  large  distributed  system,  an  industry  standard  tiered  architecture 
was  chosen.  This  architecture  pattern  defines  the  standard  tiered  system  pattern: 

Presentation  tier:  Handles  all  user  interactions  and  facilitates  that  need  to  handle  multiple  clients, 
various  visualization  paradigms,  and  abstraction  from  the  logical  tier. 

Orchestration  tier:  Web  server  endpoint  for  requirements  such  as  load  balancing,  caching, 
authentication/validation,  and  redirection. 

Services/Logical  tier:  Handles  exposing  specific  business  and  process  logic  to  facilitate  the  scalability 
needs  and  loosely  coupled  logical  components. 

Operations/Data  tier:  Exposes  all  low  level  data  and  remote  services  to  the  core  platform. 

The  main  aspects  of  the  system  will  be  described  in  the  implementation  view.  The  overall  architecture 
diagram  and  its  separation  into  design  pieces  will  be  explained. 

3.2.3.2.  Architectural  Goals  and  Constraints 

The  DARPA  project  has  placed  several  constraints  on  the  development  of  the  system.  The  following 
constraints  directly  impacted  the  architecture: 
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•  “Offerors  should  propose,  as  part  of  their  work  plan,  affirmative  steps  for  the  open  source 
promulgation  of  all  source  code,  executables,  documentation,  and  test  use  cases  for  operating 
vehic le forge  .mil.  ” 

•  “Information  assurance  controls  sufficient  to  protect  export  controlled  and  contractor  proprietary 
information  in  accordance  with  Department  of  Defense,  Department  of  State,  Department  of 
Commerce,  and  any  other  statutory  or  regulatory  requirements  or  best  practices” 

The  requirement  for  open  source  and  release  of  all  executables  to  vehicleforge.mil  has  led  to  several 
design  choices  and  architecture  modifications.  While  most  enterprise  design  patterns  are  available  in  open 
source  versions,  the  installation  and  maintenance  of  these  systems  is  challenging  due  to  lack  of  dedicated 
support  and  documentation.  The  final  deliverables  need  to  include  maintenance  and  installation  material 
and,  without  guidance  from  the  OpenSource  providers,  the  final  deliverable  will  be  as  complete  as 
possible. 

The  IA  requirements  were  defined  as  the  project  progressed.  Due  to  the  open  nature  of  the  system 
combined  with  the  need  to  include  ITAR  materials,  the  definition  of  the  controls  needed  has  morphed 
over  time.  As  controls  have  been  explained  and  refined,  the  platform  architecture  has  been  modified  to 
accommodate  these  new  constraints.  This  “as  you  go”  approach  may  cause  difficulties  in  that  some  new 
constraint  may  lead  to  drastic  changes  in  the  architecture  to  support  the  new  requirement. 

3.2.3.3.  Use-Case  View 

The  major  functionality  of  the  system  hinges  on  the  interaction  of  the  user  types  listed  below  with  the 
vehicleforge  website.  The  users  and  key  use  cases  are: 

•  Unauthenticated  user  (people  who  want  to  browse  the  website) 

•  Authenticated  user 

o  New  user 
o  Existing  user 

o  Model  builders:  someone  building  model  definitions  or  services  using  the  system 
o  Assemblers:  people  who  want  to  integrate  designs 
o  Designers:  people  who  want  to  investigate  models  to  satisfy  a  design 
o  Validators/curators:  people  who  check  the  validity  of  a  model/design 
o  Viewers:  high  level  users  who  have  a  stack  in  the  design  (PM,  business  leads) 
o  Model  providers/suppliers:  users  who  want  to  expose  existing  models/designs  to  the 
community 

o  Mentors:  users  who  answer  questions  and  provide  guidance  on  specific  topics  to  select 
users 

o  Spies:  users  who  are  trying  to  steal  IP  and  internal  design  specifications 

There  is  a  prioritized  list  of  what  these  users  will  want  to  accomplish  and  this  defined  the  feature  set  that 
needs  to  be  realized  by  the  platform.  The  feature  set  from  all  users  was  then  aggregated  and  ranked 
according  to  number  of  uses  in  all  user  categories. 

Table  1  represents  the  highest  ranked  features.  This  table  is  used  in  project  planning  to  define  the  next  set 
of  work  items. 
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Table  1.  Prioritized  list  of  features  ranked  from  highest  to  lowest 


Feature 

Rank 

Search  for  a  model  based  on  (usability/ratings/popularity/specification/ author/Free/my  own  criteria) 

17 

Comment  (make,  share,  read  reviews) 

12 

Tutorial  (how  to  build  a  model,  assemble  models,  publish  a  model,  use  the  system) 

7 

Share  and  use/publish  models 

7 

Test  bed  (industry  standard  test,  test  suite)  system 

6 

View  and  browse  all  models,  and  by  categories 

6 

Create  and  find  communities 

6 

Validation  (specific  to  a  project) 

5 

Share  (share  model  found  on  the  marketplace,  send  to,  print,  submit  result,  homework  assignment) 

5 

I  want  to  be  paid 

5 

Automate  model  connector,  3rd  plugins,  and  current  tool  integration 

4 

Tracking  models  and  get  notified  with  changes 

4 

3.2.3A  Logical  View 


The  architecture  of  the  platform  is  patterned  after  an  N-Tier  design.  This  system  has  been  broken  into  four 
tiers — presentation,  orchestration,  services,  and  operations — as  depicted  in  Figure  2.  This  design  will 
allow  for  logical  separation  of  the  platform  infrastructure  to  facilitate  an  open  and  pluggable  framework. 
The  goal  is  to  isolate  the  major  needs  of  the  website  and  create  industry  standard  links  between  them.  In 
this  way,  plugging  in  a  different  provider  of  the  feature  set  can  be  accomplished  with  minimal  effort. 
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Figure  2.  vehicleforge  architecture  overview 

The  subcomponents  identified  above  are  detailed  in  accompanying  documentation  that  explains  the 
design  decisions  and  application  links  that  facilitate  the  flow  of  data  through  the  system. 

3. 2. 3.5.  Overview 

The  details  to  each  of  the  design  areas  above  are  documented  in  accompanying  design  documents. 
OO-Overall  Design:  The  overall  design  document. 

01-Presentation  Tier:  This  document  will  describe  the  vehicleforge  website  and  design  decisions 
concerning  all  presentation  tier  development.  This  includes  the  underlying  object  model,  database  Schema 
for  core  components,  integration  with  web  components,  and  deployment  requirements. 

02-Security:  This  document  will  cover  the  security  platform  including  design  decisions  with  respect  to 
user  identity,  communication  with  TrustForge,  and  availability  of  an  apache  mod  to  facilitate  all  needed 
security  communication  for  any  included  service. 

03- Data  Storage  and  Meta  Data:  This  document  will  cover  the  data  base  software  as  well  as  the 
schema  used  in  the  core  functionality  of  the  system  as  well  as  usage  of  the  Amazon’s  Simple  Storage 
Service  (S3)  as  the  collaborative  storage  mechanism.  It  will  also  include  the  design  and  integration  with 
the  repository  software  including  storage  and  setup. 

04-Services  Tier:  This  document  will  cover  the  service  bus  architecture  with  respect  to  communications 
to  and  from  distributed  services  in  support  of  the  service  marketplace.  This  includes  design  decisions 
about  product  use  and  feature  sets  as  well  as  security  issues. 
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05-DOME  Service  Marketplace:  This  document  will  describe  the  integration  of  DOME  services  into  the 
vehicleforge  website  as  well  as  the  communication  and  security  needed  for  providers  to  supply  access  and 
restrict  access  to  their  distributed  services. 

06-Deployment  Scalability:  This  document  will  detail  the  layout  of  the  Amazon  instances  with  respect 
to  where  the  various  providers  are  provisioned  as  well  as  the  phased  approach  to  accomplishing  the  goal. 
It  will  cover  the  installation,  setup,  maintenance,  and  security  aspects  of  the  design. 

3.2.4.  Source  Information  from  Confluence 

All  the  information  from  the  Confluence  system  is  included  in  the  technical  package  submitted  as  part  of 
this  project  deliverable. 
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4. 


TECHNICAL  RESULTS 


The  main  technical  outcome  of  this  project  is  the  full  implementation  of  vehicle  forge.  This  section, 
reports  on  the  various  technical  aspects  of  this  project  including  the  vehicleforge  feature  set,  description 
of  the  main  functionality,  quality  attributes  of  the  system,  and  the  verification  and  validation  approach. 

4.1.  vehicleforge  Feature  Set 

The  website  contains  a  host  of  features  that  satisfy  each  of  the  three  tasks  stated  in  the  Statement  of  Work. 

•  Task  1  -  Develop  vehicleforge  platform  framework,  and  develop  project  hosting  and 
collaboration  components. 

•  Task  2  -  Develop  the  vehicleforge  service  marketplace. 

•  Task  3  -  Integrate  various  components  and  test  vehicleforge  platform. 

Table  2  lists  the  capabilities  and  related  features  that  were  identified  at  the  beginning  of  this  project,  and 
have  been  implemented  in  vehicleforge. 


Table  2.  A  list  of  vehicleforge  implemented  capabilities  and  related  features 


Capability 

Related  Features 

User  Management 

User  registration 

Content  moderation 

Security  and  Access  Control 

Export  control 

Customizable  permission  and  access  control 

Project  Management 

Create/update  project 

Make  project  public/private 

Branching  (public/private) 

Revision  control:  check-in  and  check-out  of  content 

Specify  keywords/tags/category 

Document  management 

Task  tracker 

Bug/Issue  tracker 

Project  activity/history 

Time  tracking  and  reporting 

Collaboration  Spaces 

Blog:  diary  feature 

Poll/survey 

Wikis 

Mailing  list 

Community  page 

Social  Networking 

Activity  stream/news  feed 

Crowd-rating  and  voting 

Commenting  (project,  model,  user) 

Tagging  project 

li 


User  profiles 

Friends /follow 

Subscriptions/messages 

Reporting  &  Analytics 

Real-time  dashboard  via  Google  Analytics 

Sharing/Co-creation 

Publish/share  project 

File  release 

Execute  model  through  web  UI 

Visual  way  to  connect  models 

Validation 

Productivity 

Search 

Site-wide  search  of  content 

Search  project  (by  name  or  category) 

Search  model  (by  name) 

Search  user  (by  name) 

Search  skills 

Search  community 

Browse 

Browse  projects 

Landing  page  to  view  projects 

View  a  project  (high  level  summary,  its  performance  and  reviews,  specification) 

Browse  services  and  components  through  project 

Association 

Graphical  view  for  association  between  components 

Additionally,  exploratory  design  has  been  conducted  to  identify  more  features  that  could  enhance  the  user 
experience,  website  utility,  and  functionality  needs  of  users.  Table  3  contains  a  list  of  features  for  which  a 
user  interface  design  has  been  developed  without  backend  functionality. 


Table  3.  A  list  of  vehicleforge  additional  features  designed 


Capability 

Related  Features 

User  management 

User  roles 

Collaboration  spaces 

Forum 

Social  networking 

Tagging  component 

Sharing/Co-creation 

Downloadable  package 

Simple  way  to  build  model  to  test  an  idea 

Integrate  models 

Plugins  for  third  party  tools 

Productivity 

Browse 

Browse  services  and  components  through  marketplace 

Association 

Association  between  related  projects 
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4.2.  Description  of  Key  vehicleforge  Capabilities 

The  vehicleforge  website  was  designed  to  reflect  the  approach  the  government  uses  to  release  design 
challenges  to  the  crowd  as  a  solicitation  mechanism  for  building  the  next  generation  military  vehicles. 
The  website  design  is  based  on  the  principles  of  online  collaboration,  networking,  and  crowdsourcing. 
The  overall  goal  is  that  members  (users  of  the  website)  will  be  able  to  team  up  with  one  another — 
regardless  of  location  and  prior  acquaintance — in  order  to  develop  virtual  project  models  that  can  be 
submitted  as  entries  to  the  design  challenges.  The  vehicleforge  website  provides  the  framework  for  this 
type  of  collaboration  as  well  as  additional  tools  and  software  to  help  build  project  ideas. 

The  following  sections  describe  the  various  elements  of  the  vehicleforge  website  and  the  corresponding 
features  implemented. 

4.2.1.  Homepage  Features 

The  homepage  is  the  vehicleforge  starting  point;  it  gives  an  overview  of  the  vehicleforge  website  and  how 
the  website  can  be  used  to  develop  project  ideas.  It  serves  to  orientate  users  to  different  parts  of  the 
website  and  gives  a  general  idea  of  website  activity.  For  a  user  who  is  new  to  the  website,  it  is  a  good 
base  to  come  back  to.  However,  as  a  user  becomes  more  accustomed  to  vehicleforge  and  is  active  within 
projects,  the  dashboard  is  recommended  as  the  base  for  website  navigation. 

4.2.11.  Current  Challenges 

Information  about  project  ideas  and  cash  prizes  for  various  challenges  is  displayed  here.  The  user  can 
read  up  on  design  challenges  that  have  been  released  to  the  public.  The  user  can  start  a  project  entry  for 
the  challenge  or  see  if  there  is  an  existing  team  to  join  based  on  what  interests  them. 

4.2.12.  Latest  News 

The  user  can  learn  about  the  most  recent  things  members  have  been  up  to  on  the  website.  By  clicking  on 
one  of  the  news  items,  the  user  can  get  more  related  information.  By  seeing  what  others  are  doing  on 
vehicleforge,  users  can  get  ideas  for  employing  the  website  and  what  activities  to  get  involved  in. 

4.2.13.  Website  Stats 

A  summary  of  whom  and  what  are  on  vehicleforge  is  shown  here.  Links  can  be  used  to  explore  different 
features  and  pages  on  the  website.  Users  are  given  the  opportunity  to  come  across  an  interesting  project  or 
connect  with  other  members. 

4.2.2.  Dashboard  Features 

The  dashboard  is  a  home  base  that  is  unique  to  the  interests  of  each  individual  user.  This  page  allows  the 
user  to  track  their  activity  on  vehicleforge.  From  the  dashboard,  it  is  easy  to  navigate  to  the  parts  of  the 
website  the  user  will  employ  the  most,  and  it  gives  quick  access  to  projects,  discussions,  and  other 
activities. 

4 .2.2.1.  Dashboard  Home 

The  purpose  of  the  dashboard  home  is  to  serve  as  a  personalized  streamed  newsfeed  where  the  user  is  able 
to  choose  to  receive  updates  on  certain  information  and  parts  of  the  website. 

•  Manage:  This  stream  displays  any  recent  activity  related  to  the  user’s  projects  and  components 
they  are  working  on.  It  allows  the  user  to  keep  track  of  what  teammates  have  accomplished  and  if 
other  members  expressed  interest  in  the  user’s  work.  Additionally,  the  side  bar  consists  of  a 
summary  of  what  the  user  is  involved  in  to  easily  manage  interests. 
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•  Subscribe:  Throughout  vehicle  forge,  there  is  the  option  of  subscribing  to  different  members, 
projects,  components,  and  services.  The  user’s  stream  contains  recent  notifications  and  updates 
relating  to  these  subscriptions.  This  allows  the  user  to  track  if  a  component  from  the  market  place 
has  been  updated  or  if  another  user  has  created  a  new  project  of  interest  to  join. 

4 .2.2.2.  Public  Profile 

The  public  profile  tab  is  where  the  user  displays  information  about  themselves  to  other  members. 

•  Edit:  By  activating  the  edit  mode,  the  user  can  change  the  profile  picture  and  create  introductory 
text. 

•  Present:  The  profile  shows  a  user’s  work  on  vehicle  forge  and  the  user’s  member  reputation.  It 
allows  users  to  connect  to  work  on  projects  together,  or  simply  make  a  new  contact. 

4.2.2.3.  Work 

The  work  tab  is  a  way  for  the  user  to  monitor  the  projects  they  are  involved  with. 

•  Artifacts:  Allows  the  user  easy  access  the  projects  they  are  a  part  of  in  order  make  changes  and 
further  the  project  design. 

•  Tasks:  Allows  the  user  to  manage  a  “to  do”  list  to  stay  on  schedule  and  make  sure  any  outlying 
tasks  are  finished. 


4. 2. 2.4.  Diary 

The  diary  tab  is  a  place  for  the  user  to  write  out  their  thoughts  while  working  on  vehicleforge.  The  user 
can  revisit  what  was  written  by  selecting  the  entry  title  at  the  bottom  of  the  page. 

•  Reflect:  The  diary  feature  can  be  used  to  make  quick  notes  about  an  interesting  component  on 
vehicleforge,  to  write  about  what  was  accomplished  on  a  project,  or  reflect  on  an  interesting 
article  relating  to  a  design  challenge. 

•  Share:  By  checking  the  “is  public”  box,  the  diary  can  be  used  as  a  sort  of  blog  or  mailing  list  that 
will  send  entries  to  members  who  are  following  the  user. 

4 .2.2.5.  Create  Project 

The  create  project  tab  is  the  portal  through  which  the  user  can  lead  a  design  team  and  build  a  virtual 
project. 

•  Grow:  The  user  has  the  chance  to  start  projects,  be  the  leader,  assemble  a  team,  and  be  recognized 
for  innovation  and  expertise. 

4 .2.2.6.  Register  Server 

The  register  server  tab  is  where  the  user  can  connect  a  vehicleforge  account  to  external  servers  to  increase 
access  to  different  resources. 

•  Expand:  By  registering  other  servers,  additional  software  and  tools  can  be  accessed  to  create  a 
more  sophisticated  final  product. 

4.2.3.  Community  Features 

The  community  page  is  the  social  hub  of  vehicleforge.  It  is  through  this  page  that  users  connect  to  peers 
and  scout  out  projects,  components,  and  challenges  to  help  develop  ideas.  Like  any  larger  social  website, 
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the  types  of  interactions  and  uses  for  the  community  page  are  limited  only  by  the  user.  The  user  is 
encouraged  to  find  creative  ways  to  connect  with  other  members  and  move  projects  forward. 


4 .2.3.2.  Community  Home 

The  home  tab  is  where  the  user  can  stay  up  to  date  with  trending  and  the  latest  challenges,  projects,  and 
components  on  vehicle  forge. 

•  Challenges:  Users  can  find  inspiration  or  a  big  cash  prize  by  getting  involved  in  a  challenge  and 
joining  in  on  the  competition. 

•  Projects:  The  latest  projects  are  likely  to  be  looking  for  members  to  recruit.  Browsing  through 
popular  projects  is  a  good  way  to  get  new  ideas  and  see  what  interesting  things  others  are  doing. 

•  Components:  Users  have  the  opportunity  to  be  the  first  to  try  out  a  latest  component  for  a  project 
or  scout  out  component  creators  to  find  an  experienced  designer  for  their  team. 

•  Sidebar:  The  sidebar  gives  the  user  quick  access  to  interesting  discussions  and  resources  that  may 
help  answer  questions  or  give  food-for-thought. 

4 .2.3.2  Members 

The  members  tab  allows  the  user  to  see  who  else  is  on  vehicleforge  and  connect  with  users  of  interest  for 
further  networking  and  communications. 

•  Search:  The  user  can  search  for  specific  members  or  by  skill  set  to  fill  a  hole  in  a  team. 

•  Browse:  The  user  can  browse  the  member  pages  to  make  a  new  connection  or  re-connect  with 
someone.  Clicking  on  a  member’s  name  activates  a  pop-up  that  describes  their  particular  skills. 

•  Learn:  By  clicking  on  the  “view  profile”  link  in  the  pop-up,  the  user  can  view  information  about 
a  member’s  recent  activity,  projects,  and  on-line  reputation. 

•  Connect:  To  join  a  member’s  project,  recruit  a  member  for  a  project,  or  just  make  a  networking 
connection,  the  user  can  leave  a  comment  on  the  member’s  profile  or  start  a  private  message 
conversation.  To  be  updated  on  a  member’s  contributions  to  vehicleforge,  the  user  can  also 
subscribe  to  that  member  and  receive  notifications  via  the  dashboard. 

4 .2.3.3.  Discussion  Board 

The  discussion  board  is  where  users  can  ask  specific  questions,  get  feedback  on  work,  or  simply  start  a 
conversation  on  a  topic  of  interest. 

•  Browse:  The  user  can  view  questions  and  conversations  other  members  are  having  or  use  the 
forum  navigation  bar  to  sort  through  specific  topics.  It  is  possible  the  user  will  find  that  their 
question  has  already  been  answered  or  may  unexpectedly  find  some  useful  information. 

•  Learn:  The  user  can  start  or  join  an  existing  discussion  to  get  questions  answered  or  share 
expertise  with  those  who  need  it. 

•  Connect:  The  discussion  board  is  a  good  place  to  meet  and  interact  with  other  members.  It  is  a 
way  for  the  user  to  thank  a  helpful  member  or  recruit  a  member  for  a  project. 


4. 2. 3.4.  Resources 

The  resource  page  is  a  member-run  center  where  users  can  find  additional  help  to  better  navigate  the 
website  and  complete  certain  actions. 
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•  Search:  The  user  can  find  information  by  searching  for  a  specific  term,  using  the  browse 
resources  links,  or  seeing  what  resources  are  most  used  or  recently  updated. 

•  Learn:  The  getting  started  and  FAQ  resource  pages  are  a  good  place  for  new  users  to  get  oriented 
with  the  website.  Experienced  users  can  use  the  technical  resource  pages  to  solve  more  complex 
problems. 

•  Teach:  The  user  can  create  a  page  or  edit  an  existing  page  to  address  a  topic  that  many  users 
might  be  having  trouble  with  or  provide  information  about  something  missing  from  the  resource 
center.  Sharing  expertise  is  a  way  for  the  user  to  boost  their  member-rated  reputation  and  find 
more  projects  to  join. 

4.2.4.  Marketplace 

The  marketplace  allows  the  user  to  reach  out  to  others  in  the  vehicleforge  community.  The  main  page 
design  was  inspired  by  GrabCAD  and  GitHub.  Users  can  share  models  and  ideas  with  others,  as  well  as 
download  other  member’s  components  for  their  own  projects.  What  differentiates  the  CEED  marketplace 
from  GrabCAD  or  GitHub  is  the  simulation  engine  “DOME”  embedded  in  the  marketplace.  The  user  can 
plug-in  and  execute  simulation  components  on  the  CEED  marketplace  to  see  how  their  design  interacts 
with  other  parts  of  the  system.  The  marketplace  is  what  makes  crowdsourcing  and  rapid  development 
possible — it  is  the  center  of  collaboration  for  vehicleforge. 

4.2.4.1  Component  Marketplace 

The  user  can  upload  a  component  for  others  to  see,  or  search  through  various  components  to  find  one  that 
could  further  aid  the  development  of  their  project.  This  will  expedite  the  project  development  process;  for 
example,  if  a  project  uses  a  CAD  to  model  a  system  that  includes  a  pulley,  it  would  be  much  easier  to  find 
one  that  was  shared  in  the  marketplace  rather  than  designing  one  from  the  beginning. 

•  Find  Components:  This  search  bar  can  be  used  to  narrow  down  the  types  of  components  within 
the  library.  After  clicking  on  a  specific  model: 

o  View  Component:  More  detail  is  given  for  the  component. 

o  Add  to  Project:  The  user  can  click  here  to  add  a  component  to  a  project. 

4.2.42.  Service  Marketplace 

The  user  can  run  simulations  to  test  different  components  in  a  virtual,  accurate  manner.  Services  are  a 
large  part  of  why  product  development  can  be  distributed  to  people  who  are  apart  from  each  other 
geographically.  Before  this  progress,  much  time  and  money  was  required  to  produce  many  prototypes; 
this  service  allows  the  cost  of  engineering  modeling  programs  to  be  cut  with  the  use  of  open  source 

software.  Without  these  simulations,  it  is  difficult  to  tell  if  the  digital  design  will  survive  the  physical 

world  with  varying  temperatures  and  loads. 

•  Fi  nd  Services:  The  user  can  narrow  a  search  to  figure  out  how  to  test  models  remotely  and  decide 
what  to  test  on  a  component  (add  dynamic  simulations,  validations,  analysis,  etc.). 

4.2.5.  Projects 

The  projects  page  is  where  the  product  development  work  is  organized.  This  is  how  the  process  of 
development  without  the  physical  presence  of  teammates  is  possible.  Once  registered,  users  have  access 
to  projects  they  are  involved  in,  as  well  as  projects  they  are  following. 
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The  user  can  click  on  a  specific  project  to  show  various  tabs  under  the  project. 

4. 2.5.1  Project  Home 

Get  an  overview  of  the  project  here. 

•  Photo  Gallery  and  Activity  section:  The  user  can  upload  photos  of  the  project  to  be  seen  by 
others  for  a  better  visualization  of  the  goal.  The  activity  section  is  where  other  members  of  the 
community  can  share  their  comments  on  the  project.  It  also  shows  the  user’s  progress  by 
publishing  different  steps  taken  on  the  project  (adding  a  new  component,  testing  it  with  a  service, 
etc.). 

•  Follow  Us:  There  are  various  ways  to  track  a  project  of  interest  online  through  other  social  media 
(namely  RSS,  Facebook,  and  Twitter)  as  well  as  on  the  vehicleforge  website  itself.  Clicking 
“Subscribe”  will  give  the  user  updates  about  that  specific  project  on  their  dashboard. 

•  Our  Team:  The  user  can  invite  other  members  to  join  a  project,  as  well  as  see  all  the  people 
involved  in  the  project  listed. 

•  Similar  Projects:  vehicleforge  presents  a  feature  where  it  orders  other  similar  projects  on  the  side 
of  the  project  home  page.  This  allows  the  user  to  seek  inspiration  by  quickly  accessing  other 
similar  projects. 

4 .2.5.2.  Components 

This  shows  an  overview  of  all  the  components  that  are  used  in  a  project. 

•  Find  Components:  If  there  are  many  components  in  a  project,  this  search  bar  allows  the  user  to 
find  a  component  efficiently.  When  a  component  is  clicked,  an  overview  of  information  is 
displayed: 

o  Files:  This  shows  documentation  on  a  specific  component  (visual  models,  text,  etc.).  The  user 
has  the  option  to  “Get  Access,”  which  allows  the  user  to  transfer  files  between  their  personal 
desktop  and  the  shared  vehicleforge  project  page. 

o  Edit:  The  user  can  change  any  information  about  the  component  here.  Additionally,  the  user 
can  add  a  sub-component  to  the  component. 

o  Services:  This  shows  the  various  services  that  have  been  tested  on  a  specific  component. 

o  Tags:  The  user  can  organize  the  component  further  by  tagging  various  features  and  uses 
within  the  component.  Later,  all  components  with  the  same  qualities  can  be  seen  listed  in  one 
place  without  having  to  go  through  all  the  components  separately. 

4 .2.5.3.  Services 

This  is  where  the  user  can  test  projects  with  various  simulations  or  tools.  (See  Component  Marketplace 
=>  Features  =>  Services.) 

•  Browse  Services:  This  is  where  all  the  services  that  have  been  used  for  a  specific  project  will 
appear.  Clicking  on  a  specific  service  will  give  for  more  information  and  detail  about  the  service. 
“Service  Subscriptions”  lists  services  from  project  components  as  well  as  from  the  marketplace. 
“Your  Services”  lists  the  services  that  the  user  has  shared  by  uploading  them  from  DOME. 

•  Register  a  Service:  This  is  where  the  user  can  upload  a  new  service  from  DOME  for  another 
type  of  analysis  on  a  certain  component.  To  register  a  service  in  a  specific  project,  the  user  must: 
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o  Choose  the  server  for  available  DOME  file 
o  Browse  and  select  the  correct  DOME  folder,  model,  then  interface 
Once  a  service  is  registered,  a  service  data  box  below  will  pop  up  that  allows  the  user  to: 

•  Share  Service:  The  user  can  share  a  service  with  others  in  the  community  through  the 
marketplace. 

•  Integrate  Services:  The  user  can  create  a  new  composite  service  by  assembling  instances  of 
existing  services,  and  then  connecting  their  input  and  output  parameters.  This  new  service  is 
called  an  “integration  service”,  and  its  internal  parts  “component  services”.  By  connecting  inputs 
and  outputs  between  component  services,  the  user  establishes  how  the  data  will  flow  through  the 
new  integration  service.  Since  all  inputs  and  outputs  must  be  accounted  for,  any  remaining  inputs 
and  outputs  from  the  component  services  determine  by  default  the  new  integration  service’s  input 
and  output  parameters. 

4.3.  Quality  and  Development  Process 

Besides  the  implemented  features  in  vehicleforge,  a  lot  of  focus  has  been  put  on  the  overall  quality  of  the 
system  in  terms  of  its  key  quality  attributes  and  the  process  by  which  it  was  architected,  designed,  and 
developed.  From  a  quality  perspective,  this  report  outlines  key  quality  attributes  that  capture  DARPA’s 
requirements  as  well  as  those  most  relevant  to  this  type  of  system.  Namely,  architecture  decisions  and 
design  choices  critical  to  the  system’s  availability,  performance,  security,  maintainability,  usability,  and 
testability  are  addressed. 

4.3.1.  Availability 

vehicleforge ’s  services  are  deployed  on  Amazon’s  cloud.  To  address  availability,  redundancy,  and 
resilience,  they  are  deployed  across  four  different  availability  zones.  For  the  purposes  of  this  project,  the 
database  management  system  is  not  redundant.  However,  the  need  to  reboot  the  database  is  mitigated 
through  a  notification  mechanism  that  alerts  the  system  administrator  that  the  database  is  down. 

To  limit  the  database  exposure  to  external  attacks,  the  following  tactic  is  adopted: 

•  Users  have  no  direct  way  to  control  database  because  it  is  deployed  behind  the  PHP  server  with 
restrictive  communication  rules. 

•  Penetration  testing  is  carried  out  to  ensure  that  no  potential  holes  exist  that  may  jeopardize  the 
system. 

The  document  storage  uses  Amazon’s  S3  (Simple  Storage  Service) — a  highly  available  storage  capability 
with  a  service  level  agreement  specifying  99.99999999999%  (11  9?s)  availability. 

The  DOME  server  is  deployed  in  such  a  way  that  it  has  no  external  access.  A  web  service  has  been 
created  to  manage  querying  status,  starting,  and  stopping  the  internal  DOME  server. 

4.3.2.  Performance 

Various  download  tests  were  performed  from  Oregon  to  Virginia  and  all  the  requirements  specified  in  the 
BAA  were  easily  exceeded.  Load  testing  with  a  single  user  and  multiple  users  (-20  user  per  Webserver, 
very  aggressive  users  with  no  think  time  between  clicks)  was  performed. 

Webserver  fail-over  tests  were  performed  and  were  all  successful.  The  auto-scaling  feature  provided  by 
Amazon’s  cloud  environment  proved  to  be  a  key  architecture  decision. 
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4.3.3.  Security 

A  key  architecture  decision  has  been  made  to  use  a  federated  access  control  mechanism  as  a  security 
tactic,  vehicle  forge  uses  Shibboleth  identity  Provider  (IdP)  to  securely  communicate  a  user  attributes  to  a 
service  provider.  It  only  passes  the  information  that  is  deemed  “required”  for  granting  access  to  the 
services  requested.  This  technology  maps  well  to  other  single  sign  on  (SSO)  system,  which  is  used 
internally  at  GE. 

Other  security  tactics  used  in  vehicleforge  include: 

•  The  website  is  HTTPS-enabled;  it  supports  encryption  of  all  connections  to  load  balancer. 

•  Inter-security  group  insulation  makes  it  so  the  system  does  not  allow  access  to  machines  from 
non-approved  sources. 

•  Data-at-rest  in  S3  buckets  is  encrypted  automatically. 

•  Almost  all  IA  controls  specified  by  government  terms  have  been  satisfactorily  implemented  in 
vehicleforge  and  in  the  supporting  infrastructure  and  services. 

4.3.4.  Maintainability 

vehicleforge  was  designed  to  be  a  highly  maintainable  system.  Consistency  from  the  requirements 
document  to  architecture  specification  to  design  artifacts  to  code  has  been  preserved.  Architecture 
documents  have  been  updated  throughout  development,  and  mostly  reflect  the  original  design.  An  ISO 
standard  for  software  design  and  architecture  documentation  was  followed.  A  support  tool  (i.e., 
Confluence)  was  used  to  manage  much  of  the  maintenance/development  documentation. 

The  delivered  technical  package  includes  read-me  files  at  all  levels  of  the  directory  structure,  step-by-step 
instructions,  and  extra  information.  Tutorial  material  has  been  developed  to  bring  new  users  and 
developers  up  to  speed  quickly  on  the  system’s  capabilities.  In  addition,  configuration  management  and 
documentation  are  also  included. 

4.3.5.  Usability 

In  terms  of  usability,  the  system  has  the  following: 

•  Context-aware  help  (tooltips  and  on-page  help)  which  removes  the  barrier  of  going  to  a  different 
page  to  find  help 

•  Brief  help,  but  can  link  to  more  detailed  information 

•  Tutorials 

•  User  experience  (i.e.,  Twitter  bootstrap  website  styling  package) 

•  Organizing  complexity  (the  dashboard  is  a  central  location  to  access  information  and  controls) 

4.3.6.  Testability 

Testability  is  important  to  ensure  that  the  system  is  changeable  and  its  use  can  evolve  as  its  users  identify 
new  ways  to  use  the  systems.  Therefore,  it  is  important  to  build  a  system  that  can  be  tested  easily  once  a 
new  change,  a  fault,  or  a  new  version  is  rolled  out.  The  team  used  a  set  of  tools  that  are  standard  within 
GE  such  as  JIRA,  Confluence,  and  Bamboo.  Historic  documentation  from  these  tools  makes  vehicleforge 
a  manageable  system  in  terms  of  testing  it. 


19 


Other  quality  assurance  techniques  have  been  employed  in  terms  of  the  process  followed  by  the  team  to 
develop  this  platform  namely: 

•  An  agile  approach  to  development  with  short  iterations  and  daily  stand-up  meetings  that  allowed 
the  team  to  uncover  and  address  design  and  code  defects  as  they  arise. 

•  Employment  of  collaboration  tools  such  as  JIRA  to  track  user  stories,  issues,  bugs,  and  backlog 
prioritization;  and  Confluence  to  share  all  related  artifacts  among  the  team  members. 

•  Continuous  integration  and  testing  enabled  by  the  Bamboo  testing  environment. 

4.4.  Deployment  and  Infrastructure  Overview 
4.4.1.  Deployment  on  the  Cloud 

Figure  3  shows  how  code  changes  are  affected  on  deployed  systems  in  the  cloud. 
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Figure  3.  High-level  overview  of  vehicleforge  deployment  on  the  cloud 


•  Developers  commit  code  changes  and  changes  to  configuration  files  on  the  code  repository 
(SVN). 

•  Stacks  can  be  started  and  stopped  by  an  administrator  from  a  deployment  console  that  has  the 
necessary  deployment  scripts  (also  in  the  repository)  and  requisite  tools/applications. 

•  The  deployment  console  will  check  all  of  the  necessary  configuration  information  out  of  the 
repository  for  a  specified  revision  number,  will  package  it,  upload  it  to  a  storage  location  (e.g.  S3 
bucket),  and  will  configure  all  of  the  cloud  resources  (via  Amazon  CloudFormation). 
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•  A  testing  console  has  also  been  developed  to  expose  some  of  this  deployment  functionality  for 
automated  tests. 

•  A  production  stack  allows  access  to  a  stable  release  of  the  website  for  users,  while  development 
and  QA  stacks  are  used  in  development. 


4.4.2.  Server  Dependency  Matrix 

Table  4  shows  the  dependencies  between  the  various  servers  that  are  deployed  as  part  of  the  vehicleforge 
system. 


Table  4.  Server  dependency  matrix 
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Xs  mark  a  dependency.  The  server  instances  listed  on  the  left  side  are  dependent  on  instances  listed  along 
the  top  if  an  X  is  present. 

Here,  a  server  type  A  is  dependent  on  server  type  B  if  A  requires  some  information  or  file  from  B  upon 
start-up.  There  is  currently  a  bi-directional  dependency  between  Shibboleth  SP  that  runs  on  the  Webserver 
and  the  Shibboleth  IdP  server.  This  is  currently  resolved  manually  after  the  stack  starts  up  by  restarting 
processes  on  each  of  these  servers. 

4.4.3.  Deployment  Process  Flow 

The  following  steps  describe  the  process  through  which  an  instance  of  vehicleforge  is  deployed  on  the 
cloud.  These  steps  are  depicted  in  Figure  4. 
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Figure  4.  vehicleforge  instance  deployment  process 

Step  1:  From  the  deployment  console,  an  administrator  initiates  the  deployment  script  that  specifies  a 
revision  number  from  the  repository  to  use  as  the  deployment  base. 

Step  2:  The  script  checks  out  the  pertinent  configuration  files  from  the  repository  and  packages  them  in  a 
S3  bucket. 

Step  3:  The  script  then  initiates  the  stack-launch  by  accessing  the  Amazon  CloudFormation  API  via  the 
command-line  tools. 

Step  4:  Amazon  CloudFormation  creates  and  initiates  the  configuration  of  several  stack  resources: 

•  PostgreSQL  Database  Instance 

•  Kerberos  Server  Instance 

•  Shibboleth  Identity  Provider  Instance 

•  DOME  Server  Instance 

•  GIT  Server  Instance 

•  Solr  Server  Instance 

•  vehicleforge  Webserver  Auto-Scaling  Group 

Step  5:  Each  instance  is  configured  using  Opscode  Chef-Solo  and  several  recipes  are  developed  for  this 
project. 
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4.4.4.  Deployment  Testing  Sequence  Diagram 

This  sequence  diagram  (Figure  5)  illustrates  the  communication  sequence  for  deployment  testing  via  a 
testing  console. 
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Figure  5.  Sequence  diagram  depicting  the  process  of  deployment  testing 
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4.5.  User  Interface  Design 

The  following  sections  contain  screenshots  of  current  and  proposed  user  interface  designs  that  went  into 
the  development  of  the  vehicleforge  website. 

4.5.1.  Home  Page 
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4.5.2.  Dashboard 
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4.5.3.  Community  Home  Page 
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4.5.4.  Members  Page  (before  Login) 
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4.5.5.  Members  Page  (after  Login) 
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4.5.7.  Discussion  Board:  Forum  (before/after  Login) 
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.  ehicle  Org©  Home  Community  Marketplace  Projects  Search.. 


Community  Home  Members  Discussion  Board  Resources 


SEARCH  RESOURCES 


BROWSE  RESOURCES 
Resources  Home 


-  Getting  Started 


Community  Features 

About  VehicleForge 
*■  Technical  Manuals 
f  FAQ 

View  All  Resources 


New  Page 


Community  Features 


The  community  page  is  the  social  hub  of  vehicleforge.  It’s  through  this  page  that  you  can  connect  to  your  peers  and 
scout  out  projects,  components,  and  challenges  to  help  you  develop  your  own  ideas.  Like  any  larger  social  site,  the 
types  of  interactions  and  uses  for  the  community  page  are  limited  only  by  yourself.  We  encourage  you  to  find  creative 
ways  to  connect  with  other  members  and  move  your  projects  forward. 

Community  Home 

The  home  tab  is  where  you  can  stay  up  to  date  with  trending  and  latest  challenges,  projects,  and  components  on 
VehicleForge. 

•  Challenges:  If  you’re  looking  for  a  little  inspiration  or  a  big  cash  prize,  getting  involved  in  a  challenge  is  a  good  way 
to  join  in  on  the  competition. 

•  Projects:  If  you  want  to  join  a  project,  the  latest  projects  are  likely  to  be  looking  for  members  to  recruit.  On  the  other 
hand,  browsing  through  popular  projects  is  a  good  way  to  get  new  ideas  and  see  what  interesting  things  others  are 
up  to. 

•  Components:  Be  the  first  to  try  out  a  latest  component  for  your  project  or  scout  out  component  creators  to  find  an 
experienced  designer  for  your  team. 

•  Sidebar:  The  sidebar  gives  you  quick  access  to  interesting  discussions  and  resources  that  may  help  answer  your 
questions  or  give  you  food  for  thought. 

Members 

The  members  tab  is  where  you  can  see  who  else  is  on  vehicleforge  and  connect  with  users  of  interest  for  further 
networking  and  communications. 

•  Search:  You  can  look  for  someone  specific  whose  work  impressed  you  or  you  can  find  members  by  skill  set  if  you 
need  to  fill  a  hole  in  your  team. 

•  Browse:  If  you’re  not  looking  for  anything  in  particular,  feel  free  to  simply  look  through  the  member  pages.  Maybe 
you’ll  make  a  new  connection  or  re-connect  with  someone  you  lost  touch  with.  To  learn  more,  click  on  a  user’s  name 
to  activate  a  pop-up  describing  particular  skills  they  have  to  offer. 

•  Learn:  By  clicking  on  the  “view  profile"  link  in  the  user  pop-up,  you’ll  be  presented  with  the  member’s  profile.  You 
can  see  what  the  user  is  up  to  underneath  the  recent  activity  heading  or  projects  box  and  view  their  on-line 
reputation  through  the  user  rating  box. 

•  Connect:  If  you  see  something  you  like  on  the  user’s  profile,  want  to  join  one  of  their  projects,  recruit  them  for  your 
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4.5.11.  Resources  Alternative:  View  Al I 


Shown  only  after  login 


4.5.12.  Marketplace 


Vehicle  FOrge  Home  Community  Marketplace  Projects  Search... 


Z'  ®  #  Na  Cheng  ▼ 


Welcome  to  the  Marketplace! 

your  one-stop  shop  for  shared  blueprints,  models,  concepts  &  code 


0  Share  Component 


What  are  you  looking  for? 


Q.  Search 


COMPONENT  TAGS 

Most  Popular  Recently  Added  My  Projects 

CAD  model  Java  script 

simulation  Matmab 

autorrotvc  formula  prototyping 
visualization  ifab  signals 

150  x 103 

150 x 103 

concept  mechanical 

hinges  data 

rotor 

Model  Name 

Model  Name 

Creator's  Name 

Creator's  Name 

FILTER  COMPONENTS 

47  subscribers  |  3 

47  subscribers  |  3 

Domain  Area 

-  ■ 

comments 

comments 

All 

108 

Electrical 

hydraulic 

14 

150x103 

150x103 

Mechanical 

Thermal 

6 

Electromagnetic 

3 

Model  Name 

Creator's  Name 

Model  Name 

Creator’s  Name 

Aero 

1 

47  subscnbers  |  3 

47  subscribers  1 3 

Software 

32 

comments 

comments 

Category 

+ 

150x103 

150x103 

Model  Name 

Model  Name 

Creator's  Name 

Creator’s  Name 

47  subscribers  |  3 

47  subscribers  |  3 

comments 

comments 

Creator's  Name 
47  subscribers  |  3 
comments 


32 


4.5.13.  Marketplace  (Alternative  View) 


Welcome  to  the  Marketplace! 

your  one-stop  shop  for  shared  blueprints,  models,  concepts  &  code 


CS  Share  Component 


What  are  you  looking  for? 


Q.  Search 


COMPONENT  TAOS 

CAD  model  Java  script 

simulation  MathLab 

automotvo  formula  prototyping 
visualization  Fab  signals 
ooncopt  mechanical  rotor 
hinges  data 

FILTER  COMPONENTS 


All  102 

Electrical 

hydraulic  i« 

Mechanical  «i 

Thermal  e 

Electromagnetic  3 

Aero  i 

Software  32 


Most  Popular  Recently  Added  My  Projects 


W‘ 


r* 


Category 


^ i 


m 


Model  Name 

Creator's  Name  |  updated  May  29. 2012  12PM  1 47  subscribers  1 3  comments 
This  is  the  description  of  the  model. 

Model  Name 

Creator  s  Name  |  updated  May  29.  2012  12PM  1 47  subscribers  |  3  comments 
This  is  the  description  of  the  model. 

Model  Name 

Creator  s  Name  |  updated  May  29, 2012  12PM  1 47  subscribers  1 3  comments 
This  is  the  description  of  the  model. 

Model  Name 

Creator's  Name  |  updated  May  29,  2012  12PM  |  47  subscribers  |  3  comments 
This  is  the  description  of  the  model. 

Model  Name 

Creator  s  Name  |  updated  May  29.  2012  12PM  1 47  subscribers  1 3  comments 
This  Is  the  description  of  the  model. 

Model  Name 

Creator  s  Name  |  updated  May  29.  2012  12PM  1 47  subscribers  |  3  comments 


Add  to  Project 


Add  to  Project 


Add  to  Project 


Add  to  Project 


Add  to  Project 


Afifi  In  Dmiarl 
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4.5.14.  Component  Page 


Details  Files 

Services  Sub-components 

Projects 

Name 

Description 

Last  Updated 

Concept 

Describes  the  powertrain  simulator 

Jan  4. 2012 

Guide 

Guide  to  use  the  simulator 

Jan  4. 2012 

3D  Model 

Model  you  can  use  in  the  simulator 

Jan  4. 2012 

Math  Lab  Script 

Simulation  script 

Jan  4. 2012 

Details 

Files  Services  Sub-components 

Projects 

Namo 

Description 

Last  Updatod 

Torque 

Simulates  the  torque  on  the  powertrain 

Jan  4.  2012 

|U  +  Create  Service  J 

Details  Files 

Services  Sub-components  Projects 

Name 

Description 

Aardvark8 

This  is  a  demo  project  to  show  the  capabilities  of  the 
system. 

GE  Vehicleforge 
Development 

This  project  will  track  the  work  being  done  by  the 
development  group  to  build  this  site.  Watch  this  site  to 
see  development  efforts,  tasks,  and  progress  in 
competing  the  work. 

Details  Files 

Services  Sub-components 

Projects 

Name 

Description 

Last  Updated 

Powertrain  Model 

Model  of  the  powertrain 

Jan  4.2012 

1 

+ Create  Subcomponents  J 
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5. 


IMPORTANT  FINDINGS  AND  CONTRIBUTIONS 


5.1.  Contributions 

In  this  project,  a  web-based  collaborative  crowdsourcing  environment  has  been  developed  to  democratize 
the  design  process  of  complex  CEM  systems.  The  CEED  platform  is  constructed  to  achieve  six  goals: 

1.  Provide  a  collaborative  design  environment  where  CEM  systems  can  evolve. 

2.  Address  system-level  integration  of  components  represented  by  cross-platform,  multi-domain, 
multi-physics  models,  analysis,  and  simulations  with  a  distributed,  declarative,  and  emergent 
integration  paradigm. 

3.  Attract  a  crowd  of  productive  experts,  non-experts,  and  companies  to  a  cooperative  and 
rewarding  design  ecosystem. 

4.  Democratize  the  hardware  design  process  by  providing  tools,  test  environments,  and  reiterative 
constructive  criticism  that  drives  selection. 

5.  Facilitate  design-cycle  incorporation  of  real-world  performance  feedback  received  from  dynamic, 
competitive,  or  hostile  environments. 

6.  Enable  IP  protected  sharing  in  a  collaborative  environment  for  sustainable  business  models. 

The  CEED  platform  has  integrated  project  hosting  and  management,  an  integrated  simulation  engine, 
distributed  service  marketplace,  collaboration,  security,  search,  web  analytics,  and  user  experience 
components  based  on  customization  and  extension  of  existing  open  source  solutions.  Through  these 
constructs,  benefits  in  the  context  of  crowdsourcing  hardware  design  are  being  enabled,  as  described  in 
Table  5: 


Table  5.  Key  elements  of  the  CEED  platform 


Key  Elements 

Benefits 

Project  hosting  and  repository 

•  Allow  geographically  distributed  project  development 

Distributed  service  marketplace 

•  Sharing  models  and  resources  as  services 

•  Lower  entry  barrier 

•  Rapid  dissemination  of  technology  development 

Integrated  simulation  environment 

•  Modular  design  reuse 

•  Allow  integration  of  computational  processes  and  capabilities 

Crowd-oriented  design  and  synthesis 
activities 

•  Democratize  design  and  manufacturing 

•  Collaboration  through  social  networks 

•  Continuously  seed,  sustain,  and  grow  the  hardware  design  crowd 

Sustainable  business  model  (open, 
proprietary,  classified) 

•  Protect  IP  core 

•  Bring  suppliers  and  customers  closer 

•  Commoditize  low  value  activities 

Information  assurance  and  certification 

•  Protect  against  malicious  contribution 

•  Protect  data  while  supporting  availability 
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5.2.  Important  Findings 

Finding  #1:  Integrated  simulation  capability  is  needed  in  addition  to  a  forge 

Establishing  a  robust  project  hosting  and  model  repository  fulfills  many  of  the  requirements  for 
supporting  collaboration.  However,  the  overarching  goal  of  democratizing  CEM  design  requires  highly 
responsive  modeling  and  simulation  systems  that  move  customers  and  suppliers  closer  together,  allowing 
for  rapid  creation,  validation,  and  delivery  of  products  that  closely  match  individual  preferences.  To  that 
end,  a  project  hosting  and  model  repository  only  solution  is  limited. 

One  of  the  salient  features  of  the  CEED  platform  is  a  simulation  “engine”  (DOME)  that  contributors  use 
to  plug  in  and  test  simulated  components  in  an  IP  protected  fashion.  Users  can  attach  simulation  services 
to  explore  the  behavior  of  complex  systems,  and  to  predict  problems  earlier  to  get  a  better  design  faster. 
In  addition,  although  through  META  and  iFAB  DARPA  is  releasing  a  new  set  of  design/manufacturing 
tools  to  be  available  for  use  by  the  general  design  community,  there  are  still  many  computer-aided  design 
tools  being  used  that  are  difficult  to  use  and  expensive.  With  DOME,  the  vehicleforge.mil  program  will 
allow  solo  inventors  or  small  teams  to  tap  into  those  capabilities. 

Finding  #2:  Marketplace  as  a  construct  for  sustainable  crowdsourcing 

A  collaborative  environment  allows  designers  to  team  up  to  develop  hardware  components  concurrently. 
It  also  promotes  an  active  evolutionary  design  process  capable  of  sourcing  desired  components.  Interested 
members  of  the  crowd  can  test  shared  components  and  collaborate  through  the  marketplace  in  the  system. 
The  openness  of  the  marketplace  will  allow  input  from  a  wide  range  of  contributors  with  varying  skill 
levels.  Some  contributors  will  be  experienced  model  designers  who  rigorously  test  a  model  before  making 
it  available  in  the  marketplace.  Others  will  be  less  experienced  designers  who  do  not  have  the  time  or 
expertise  required  to  test  their  contribution.  When  these  models  are  contributed  to  the  marketplace,  they 
will  experience  market  pressures  that  will  distinguish  well-designed  models  from  the  rest. 

As  the  marketplace  evolves,  the  models  in  the  marketplace  will  increase  in  complexity  and  it  will  become 
more  competitive.  Individual  models  will  compete  to  be  integrated  into  other  models.  The  competition 
among  models  will  encourage  model  contributors  to  improve  their  designs  to  suit  market  needs.  Over 
time,  lower-level  models  will  become  more  stable  and  full-featured  higher-level  integrated  models  will 
emerge.  The  competition  in  the  marketplace  will  lower  the  costs  required  to  design  and  integrate  models, 
which  will  drive  down  the  overall  cost  of  development  while  improving  outcomes. 

In  addition,  the  marketplace  allows  various  business  models  to  be  deployed.  Participants  can  be 
compensated  for  the  models/data  shared  on  the  marketplace,  and  for  providing  reviews  or  testing  services, 
etc.  Using  the  service  marketplace,  product  developers,  small  or  large,  can  flexibly  subscribe  to  and  inter¬ 
relate  services  through  appropriate  computer-mediated  interactions,  building  service  network  models  that 
embody  the  capabilities  of  a  product  development  organization. 

Finding  #3:  IP  protection  mechanism  during  crowd  collaboration 

The  acceptance  of  the  marketplace  as  a  viable  location  to  search  for  and  find  high-quality  model  services 
is  enhanced  when  proprietary  models  participate.  The  nature  of  a  proprietary  model  is  such  that  publicly 
sharing  its  intimate  details  decreases  its  value,  and  allows  vulnerabilities  to  be  found  and  exploited. 
Therefore,  contribution  of  proprietary  models  to  the  marketplace  is  discouraged  by  a  model’s  owner. 
However,  the  value  of  a  proprietary  model  will  increase  as  it  is  integrated  into  other  successful  models,  as 
its  designer  can  profit  from  integration.  If  the  value  of  a  proprietary  model  can  be  preserved  and 
additional  value  can  be  added,  due  to  integration  into  or  with  other  models,  then  it  is  in  the  designer’s 
interest  to  participate  in  the  marketplace. 
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The  CEED  vehicleforge  marketplace  protects  proprietary  knowledge.  When  a  model  is  published  through 
DOME,  its  contents  are  encapsulated  and  only  interfaces  of  desired  functions  are  exposed.  This  allows 
developers  with  proprietary  restrictions  to  share  the  model  as  a  black  box  so  that  others  can  use  the 
functionality  of  the  model  over  the  Internet.  This  is  important  because  it  enables  the  inclusion  of 
proprietary  models.  It  allows  companies  with  intensive  design  and  manufacturing  expertise  to  participate 
and  contribute  without  giving  away  proprietary  knowledge,  and  also  allows  them  to  benefit  from 
collaboration  with  the  crowd.  This  mechanism  will  also  benefit  individuals  and  small  businesses  since 
they  then  could  leverage  the  design  services  that  are  beyond  their  own  expertise  and  gain  access  to  tools 
to  develop  new  products.  Therefore,  the  marketplace  not  only  protects  a  proprietary  model’s  value 
through  encapsulation,  but  it  can  also  increase  its  value  by  safely  exposing  it  to  a  crowd. 

Finding  #4:  vehicleforge  CEED  platform  to  support  the  AVM  design  process 

The  Adaptive  Vehicle  Make  (AVM)  program  is-at  its  core-an  exercise  in  the  collaboration  of  multiple 
design  and  manufacturing  disciplines.  This  includes  META  teams  defining  models,  iFAB  teams  defining 
manufacturing  processes,  and  vehicleforge  teams  supporting  the  democratization  of  design.  One 
challenge  of  being  part  of  the  AVM  performer  community  is  that  many  of  AVM  efforts  are  dependent  on 
each  other  (i.e.  C2M2L  and  META),  but  need  to  evolve  on  their  own  for  a  while  due  to  the  parallel  nature 
of  the  program  schedule.  Existing  collaboration  channels  mainly  have  idea-exchange  focuses  (i.e.  PI 
meetings,  and  knowledge  sharing  through  the  share  point.)  Therefore,  there  is  a  potential  opportunity  for 
additional  collaboration  channel  in  the  AVM  community  and  ultimately  the  Fast  Adaptable  Next- 
Generation  Ground  Vehicle  (FANG)  challenge  crowds. 

DOME  provides  a  platform  upon  which  a  community  can  seed,  sustain,  and  grow  a  collection  of  crowd 
contributed  models  and  services  (Figure  6).  The  GE/MIT  vehicleforge  platform  may  be  a  good  channel  to 
enable  the  AVM  performers  to  expose  their  models  through  a  service  marketplace.  It  can  provide  open 
access  of  all  the  AVM  team’s  models/processes  through  DOME  before  its  maturation,  and  provide  more 
possibility  for  synergy. 


META  Toot  Chain '■ 
And  workflow 


META 

mode] 


.Traditional 

Model 

^Existing  IP  core 


META 

model 


META 

model 


IFAB  Models 


Foundry 


TradlionaV 
Modul  1 
/Existing  [Pi 


Figure  6.  DOME  defines  engineering  constructs  necessary  to  connect  engineering  and  manufacturing  tools 


37 


Finding  #5:  Culture  challenges  appose  introduction  of  crowdsourcing  in  controlled  corporate 
environments 

Summaries  of  findings  through  voice-of-the-customer  (VOC)  meetings  with  GE  engineers  working  on 
aircraft  engine  design: 

•  Compared  with  crowdsourcing  and  grass  roots  style  approach  to  build  systems,  the  engineers  are 
more  comfortable  with  the  traditional  “V”  type  of  systems  engineering.  Most  of  the  engineers’ 
reasonability  falls  within  component  level;  a  cross-components  systems  level  view  is  not  shared 
by  all.  Chief  engineers  want  control  points  of  the  design  workflow,  instead  of  fully  automated. 

•  The  CEED  platform  is  a  new  way  of  sharing  design  data  and  models.  In  current  practice,  each 
design  iteration  results  in  many  local  copies  of  design  data  and  artifacts  where  sharing  is  done 
through  manual  hand-off  processes,  causing  mistakes  and  delays.  Model  repository  is  one  method 
to  address  that.  The  engineers  also  found  the  CEED  marketplace  to  be  a  productive  way  to  share 
and  search  design  models,  history,  and  knowledge. 

•  Web-based  platforms  are  the  future  trend  for  engineering  software  tools,  but  there  is  still  a  good 
population  of  engineers  who  prefer  UNIX  shell,  command  line  environments,  and  other 
traditional  methods.  Therefore,  the  challenge  is  to  provide  a  collaborative  design  environment 
solution  in  a  platform  agnostic  way,  and  customize  it  to  serve  the  needs  of  a  mix  of  cultures. 
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6. 


IMPLICATIONS  FOR  FURTHER  RESEARCH 


Crowdsourcing  platforms  achieve  success  only  to  the  extent  that  they  are  able  to  attract  good  individuals 
and  motivate  them  to  perform  adequate  work  towards  the  end  goals  of  the  community.  As  a  future  work, 
the  project  team  will  explore  a  systematic  way  to  build  in  incentive  mechanisms  and  constructs  to  attract 
users  and  encourage  contribution  of  high  quality  models  to  vehicleforge.miL 
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7.  STANDARD  FORM  298,  AUGUST  1998 

See  last  page  of  this  report. 
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8. 


TEAM  BIOS 


The  following  arethekeypersonnelofthevehicleforge.mil  team. 

Dr.  Qing  Cao 

Dr.  Cao  is  a  lead  researcher  in  the  Software  Sciences  and  Analytics  organization  at  GE  Global  Research. 
Dr.  Cao  has  six  years  of  industry  R&D  experience  building  solutions  for  intelligent  analysis,  design,  and 
operation  of  complex  networked  physical  and  digital  systems.  Dr.  Cao’s  research  focuses  on  collaborative 
design  methodology,  integrated  simulation,  product  development  process,  search,  asset  tracking,  and 
other  decisioning  methodologies,  with  applications  in  broad  business  domains,  including  aviation,  wind, 
energy  services,  healthcare,  etc.  Dr.  Cao  is  active  in  her  research  community  and  organizes  symposiums 
for  the  ASME  Manufacturing  Science  and  Engineering  Conference.  Dr.  Cao  received  her  Ph.D.  in 
Mechanical  Engineering  from  Massachusetts  Institute  of  Technology  in  2006,  specializing  in  design 
methodology.  She  received  her  B.S.  and  M.S  in  Automated  Control  from  Tsinghua  University  in  China  in 
1997  and  2000,  respectively. 

Dr.  David  R.  Wallace 

Dr.  David  R.  Wallace  is  a  professor  of  Mechanical  Engineering  Director  of  MIT’s  Computer-Aided 
Design  Laboratory  and  a  MacVicar  Faculty  Fellow  at  MIT.  Dr.  Wallace  obtained  his  Ph.D.  from  MIT  in 
Mechanical  Engineering  in  1994.  Since  that  time,  he  has  taught  design  theory  and  methodology  at  MIT. 
He  has  been  the  principal  investigator  of  several  key  initiatives  in  industrial  design  and  product 
development  sponsored  by  the  National  Science  Foundation,  Alliance  for  Global  Sustainability,  Ford 
MIT  Alliance,  MIT  Center  of  Innovation  in  Product  Development,  and  MIT  Leaders  for  Manufacturing 
Program.  His  research  interests  include  environmentally-conscious  product  design,  integrated  computer- 
aided  design,  industrial  design  and  aesthetics,  visual  communication,  and  product  design  and  new  media 
education.  Dr.  Wallace  has  authored  more  than  50  scientific  articles.  He  serves  on  the  International 
Society  of  Industrial  Ecology  and  American  Association  of  Computing  Machinery  special  interest  group 
in  Evolutionary  Optimization. 

Dr.  Benjamin  Beckmann 

Dr.  Benjamin  Beckmann  is  a  Computer  Scientist  in  the  Business  Integration  Technologies  Lab  at  the  GE 
Global  Research.  He  received  BS  (2002)  and  MS  (2004)  degrees  in  Computer  Science  from  Western 
Michigan  University,  and  his  Ph.D.  (2010)  in  Computer  Science  from  Michigan  State  University.  His 
research  in  the  areas  of  evolutionary  computation,  cloud  computing,  crowdsourcing,  self-organizations, 
and  artificial  life  are  motivated  by  his  desire  to  understand  interactions  among  individual  entities  in  a 
complex  virtual,  cyber-physical,  and  social  systems. 

Dr.  Beckmann  has  established  a  strong  track  record  as  an  independent  researcher  with  a  steady  flow  of 
publications  ranging  from  the  evolution  of  quorum  sensing  in  self-replicating  computer  programs  to 
adaptive  logic  for  balancing  of  non-functional  tradeoffs  during  system  reconfiguration.  His  research  has 
focused  on  evolutionary  pressures  to  form  communities  and  mold  behavior.  He  has  been  an  active 
contributor  to  open-source  software  platforms  (including  Avida,  PECL,  and  the  forthcoming 
vehicleforge)  that  support  his  research.  His  research  has  led  to  twenty  refereed  publications,  and  has  been 
the  focus  of  articles  in  The  New  York  Times,  New  Scientist,  and  World  of  Intelligence  (a  French 
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